jc949 U.S. PTO 



10/26/00 



07) 



Attorney Docket No. . 



16869P-01 72001 JS 



"Express Mail" Label No. . 
Date of Deposit: 



EL381322182US 



.CO 

October 26. 2000 s 5g ^ 



Customer No. 20350 

TOWNSEND and TOWNSEND and CREW LLP 
Two Embarcadero Center, 8 th Floor 
San Francisco, California 941 1 1-3834 
(650) 326-2400 

ASSISTANT COMMISSIONER FOR PATENTS 
BOX PATENT APPLICATION 
Washington, D.C. 20231 

Sir: 

Transmitted herewith for filing under 37 CFR 1.53(b) is the 
[ X ] patent application of 
[ ] continuation patent application of 
[ ] divisional patent application of 
[ ] continuation-in-part patent application of 

Inventor(s)/Applicant Identifier: Mineo Nomoto, Daiske Katsuta, Toshio Asano, Kaoru Sakai, Tetsuo Taguchi, and Isao 
Tanaka 



so 



Assistant Commissioner for Patents 
Washington, D.C. 20231 



cm*-- — 



50 



For: Nondestructive Inspection Method and Apparatus 

[ X ] This application claims priority from each of the following Application Nos./filing dates: 



Japanese Patent Application No. 1 1 -303480. Filed October 26. 1 999 



the disclosure(s) of which is (are) incorporated by reference. 
Please amend this application by adding the following before the first sentence: "This application is a [ ] continuation [ ] 

continuation-in-part of and claims the benefit of U.S. Application No. 60/ , filed , the disclosure 

of which is incorporated by reference." 



Encl©sed are: 



t 

[ i-s 

[ r 
-[ r 

[ 

[ ]Q 

[x:i; 

NO 

[ ]Q 
[ ] w - 



16 page(s) of specification 

_5 page(s) of claims 

_J page of Abstract 

J7 sheet(s) of [ X ] formal 



[ ] informal drawing(s). 
Hitachi. Ltd. 



An assignment of the invention to . 
A [ ] signed [ ] unsigned Declaration & Power of Attorney 
A [ ] signed [ ] unsigned Declaration. 

A Power of Attorney by Assignee with Certificate Under 37 CFR Section 3.73(b). 

A verified statement to establish small entity status under 37 CFR 1.9 and 37 CFR 1.27 [ ] is enclosed [ ] was filed in the 
prior application and small entity status is still proper and desired. 

A certified copy of a Japanese application. 

Information Disclosure Statement under 37 CFR 1.97, Form PTO- 1449, and four references. 
A petition to extend time to respond in the parent application. 

Notification of change of [ ] power of attorney [ ] correspondence address filed in prior application. 





(Col. 1) 


(Col. 2) 


SMALL ENTITY 




OTHER THAN 
SMALL ENTITY 


FOR: 


NO. FILED 


NO. EXTRA | 


RATE 


FEE 


OR 


RATE 


FEE 


BASIC FEE 








$345.00 


OR 




$710.00 


TOTAL 
CLAIMS 


29 -20 


= *9 




x $9.00 = 




OR 


x $18.00- 


$162.00 


INDEP. 
CLAIMS 


8 -3 


*5 




x $39.00 = 




OR 


x $80.00 = 


$400.00 


f 1 MULTIPLE DEPENDENT CLAIM PRESENTED 




+ $130.00 = 




OR 


+ $260.00 = 




* If the difference in Col. 1 is less than 0, enter "0" in 
Col. 2. 


TOTAL 




OR 


TOTAL 


$1,272.00 



Please charge Deposit Account No. 20-1430 as follows: 

[ X ] Filing fee $ 1.272.00 

[ X ] Any additional fees associated with this paper or during the pendency of this application. 
[ ] The issue fee set in 37 CFR 1.18 at or before mailing of the Notice of Allowance, pursuant to 37 CFR 
1.311(b) 



[ 1 



A check for $ 



is enclosed. 



2 extra copies of this sheet are enclosed. 



Respectfully submitted, 
I0^NSlT 



Telephone: 
(650) 326-2400 



Facsimile: 
(415) 576-0300 




Paul A. Durdik 
Reg No.: 37,819 
Attorneys for Applicant 



PA 3105971 v1 



Attorney Docket No. 16869P-017200 
ClientRef.No. 219901021US1 



PATENT APPLICATION 



Nondestructive Inspection Method and Apparat 



us 



Inventors: Mineo Nomoto 

Yokohama, Japan 
Citizenship: Japan 

Daiske Katsuta 

Yokohama, Japan 
Citizenship: Japan 

Toshio Asano 

Hiroshima, Japan 
Citizenship: Japan 

Kaoru Sakai 

Yokohama, Japan 
Citizenship: Japan 

Tetsuo Taguchi 

Hitachi, Japan 
Citizenship: Japan 

Isao Tanaka 

Tokai, Japan 
Citizenship: Japan 



Assignees: Hitachi, Ltd. 

6, Kanda Surugadai 4-chome 
Chiyoda-ku, Japan 
Incorporation: Japan 



Entity: Large 



TO WNSEND and TOWNSEND and CREW LLP 
Two Embarcadero Center, 8 th Floor 
San Francisco, California 941 11-38^4 
(415)576-0200 



1 



PATENT 

Attorney Docket No. 16869P-017200 
Client Ref. No. 219901021US1 

5 

Nondestructive Inspection Method and Apparatus 



CROSS-REFERENCES TO RELATED APPLICATIONS 
This application claims priority from Japanese Patent Application 
10 Reference No. 1 1-303480, filed October 26, 1999. 



BACKGROUND OF THE INVENTION 
The present invention relates to a method for inspecting a crack in a metal 
surface or the like, and particularly, to an inspection method and apparatus for 
1 5 nondestructive inspection such as liquid penetrant inspection and magnetic particle 
testing. 

Liquid penetrant inspection and magnetic particle testing are 
nondestructive inspection methods for inspecting a flaw such as a fracture (crack) in a 
metal surface. In the liquid penetrant inspection, a red liquid ("penetrant") is typically 

20 applied to an inspected surface. The penetrant is wiped off after a predetermined period, 
and a white powder ("developer") is applied to the surface. If there is a flaw such as a 
crack, the developer draws the penetrant remaining in the crack to the surface, revealing 
the location, shape and size of the flaw. 

In magnetic particle testing, a solution that includes fluorescent magnetic 

25 powder is applied to a magnetic object under inspection; then the object is magnetized. If 
there is a flaw such as a crack, the magnetic flux concentrates on the flaw, attracting the 
fluorescent magnetic powder. Under ultraviolet light, the magnetic powder emits green 
light which outlines any flaw. Conventionally, verification of these flaw indicators has 
been only by visual observation. 

30 In the case of conventional inspection by visual observation, a flaw may be 

overlooked because of an inspector's fatigue, or the inspection results may differ due to 
differences of perception or opinion among individual inspectors. Further, the inspection 
result is recorded merely as a word, "passed" or the like. Thus, there is a problem in 
reliability of inspection. 
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Moreover, in magnetic particle testing, an automatic inspection apparatus 
has been developed for use with important and mass-produced parts; however, the 
apparatus is specialized, and various shaped parts cannot be readily inspected. 

Further, in liquid penetrant inspection surface colors must be detected as a 
5 highly precise two-dimensional distribution. Even if a colorimeter capable of high 
precision chromaticity measurement is used, two-dimensional sweeping is necessary. 
Thus, it is difficult from the standpoint of inspection time and cost to automatically 
inspect various shaped parts. 

In addition, when an object under inspection is large, it may be difficult to 
10 distinguish the part of the object corresponding to an image obtained by automatic 
inspection, or to discern the part of the object corresponding to a detected flaw. 

Further, the economical value will be dramatically improved if both of the 
liquid penetrant inspection and the magnetic particle testing could be performed 
automatically by one apparatus. 

15 

SUMMARY OF THE INVENTION 
An object of the present invention is to provide a flaw inspection method, 
a flaw inspection apparatus, and a flaw inspection assisting method that solve the above- 
described problems and facilitate determination of true flaws. 

20 Another object of the present invention is to provide a flaw inspection 

method, a flaw inspection apparatus and a flaw inspection assisting method that readily 
discerns the location of a flaw, even in a large object under inspection. 

In an embodiment of the present invention, to attain the above objects, a 
color video camera is used to perform image pickup on an object under inspection. Note 

25 that if an unmodified color video camera is used in liquid penetrant inspection, image 
pickup cannot be properly performed because of reflected light from the illuminated 
object. Further, in magnetic particle testing, all the foreign materials and the like on the 
object under inspection emit light due to illumination by ultraviolet light, making 
differentiation between these materials and the flaws difficult. Accordingly, a polarizing 

30 filter is provided in both luminaire and camera. Further, in a specific embodiment of the 
present invention, to remove the reflection of light, and, to reduce the ultraviolet light, a 
filter is provided in front of the camera. 



3 



Further, in the embodiment of the present invention, since the color 
camera, a white light and an ultraviolet light are provided in one probe, the probe can be 
used in both liquid penetrant inspection and magnetic particle testing. In liquid penetrant 
inspection, the xy chromaticity of the surface of the object under inspection is calculated 
5 from a video signal from the color video camera, to detect a red flaw indication. In 

magnetic particle testing, differentiation processing is performed on a green video signal 
so as to enhance and detect a flaw. 

Further, to prevent oversight and over-detection in automatic inspection, in 
one embodiment of the present invention the inspection result is displayed as a color 

10 image, and a portion judged as a flaw in the automatic inspection is surrounded by a 

rectangle so that an inspector can check each rectangular portion in an original image to 
determine whether it is a true flaw. The original image and the inspection result are 
stored as a record in a magneto-optic disk or the like. 

Further, in one embodiment of the present invention, if the object under 

1 5 inspection is, e.g., long and cannot be seen within the viewing screen, a scale is provided 
in an image pickup viewing screen, and image pickup is simultaneously performed on the 
scale and an inspection image to specify the inspection position. 

Further, in the embodiment of the present invention, in the method based 
on liquid penetrant inspection, color calibration of the color video camera employs 

20 conversion parameters for xyY values unique to the color video camera, which are 

generated by using signals obtained by image pickup with the color video camera for R 
(red), G (green), B (blue) and W (white) reference colors. An image obtained by image 
pickup on the surface of the inspected object is temporarily converted with the 
conversion parameters, and a potential flaw is detected by using the converted image. 

25 These and other objects, features and advantages of the embodiments of 

the present invention will be apparent from the following detailed description of the 
preferred embodiments of the invention, as illustrated in the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
30 Fig. 1(a) is a perspective view showing an example of an object under 

liquid penetrant inspection according to one embodiment of the present invention, and 
Fig. 1(b) is a perspective view showing an example of an object under magnetic particle 
testing according to one embodiment of the present invention. 
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Fig. 2 is a front view of the schematic construction of the flaw inspection 



apparatus showing one working example according to an embodiment of the present 
invention. 

Figs, 3(a) and (b) are front views of the monitor display screen showing 
5 the effect of a polarizing filter. In Fig 3(a) a filter is not attached; in Fig. 3(b) a filter is 
attached. 

Figs. 4(a) and (b) are front views of the monitor display screen showing 
the effect of the ultraviolet light reduction filter. In Fig. 4(a) a filter is not attached; in 
Fig. 4(b) a filter is attached. 
10 Fig. 5 is a flowchart showing the flow of the automatic inspection method 

in a liquid penetrant inspection according to an embodiment of the present invention. 



chromaticity value of the largest number of pixels; and Fig. 9(c) is a y chromaticity value 
of the largest number of pixels. 

Fig. 10 (a) is a diagram indicating chromaticity at position (i j) of an 
image; Fig. 10(b) is an xy chromaticity plane, or chromaticity diagram; and Fig. 10(c) is a 
20 diagram indicating hue at position (i j) of an image. 

Fig. 1 1(a) is a diagram indicating chromaticity at position (ij) of an image; 
Fig. 1 1(b) is an xy chromaticity plane, or chromaticity diagram, indicating a color 
difference; and Fig, 1 1(c) is a diagram indicating hue at position (ij) of an image. 

Fig. 12 is an xy chromaticity plane showing the method for obtaining a 
25 flaw candidate area. 

Fig. 13(a) is a diagram showing a flaw candidate area; Fig. 13(b) is a 
diagram showing a color difference with respect to the reference white on 132 in Fig. 
13(a); Fig. 13(c) is a diagram showing a flaw area; and Fig, 13(d) is a diagram showing a 
color difference differentiation distribution. 
30 Fig. 14 (a) indicates a judgment line 144a in a color difference and color 

difference differentiation plane where a flaw and a pseudo flaw are apparently 
distinguishable, and Fig. 14 (b) indicates a judgment line 144b in a color difference and 
color difference differentiation plane where a flaw and a pseudo flaw are not separated. 
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Fig. 
Fig- 
Fig, 
Fig. 



6 is an xy chromaticity diagram. 

7 is a front view of the construction for camera calibration. 

8 is a flowchart showing the flow of camera calibration processing. 

9 (a) is a two-dimensional chromaticity histogram; Fig. 9(b) is an x 
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Fig. 15 is a flowchart showing an example of an image processing 
algorithm for magnetic particle testing according to an embodiment of the present 
invention. 

Fig. 16 (a) is an image of luminance distribution of a flaw and apseudo 
5 flaw; Fig. 16(b) is a luminance distribution on line 161 in Fig. 16(a); Fig. 16(c) is an 
image of a flaw; and Fig. 16(d) indicates a judgment threshold value 164 in a 
differentiation diagram. 

Fig. 17 is a flowchart showing the process of flaw verification and data 
storage in an embodiment of the present invention. 
10 Fig. 1 8 is a diagram showing an example of the method for generating a 

flaw candidate marker in an embodiment of the present invention. 

Fig. 1 9 is a diagram showing an example of a flaw candidate display 
method in an embodiment of the present invention. 

Fig. 20 is a diagram showing an example of the method for specifying an 
1 5 inspection position according to an embodiment of the present invention. 

Fig. 21 is a diagram showing an example of an inspection image, including 
information specifying an inspection position according to an embodiment of the present 
invention. 

Fig. 22 is a diagram showing an example of the structure of inspection 
20 result data stored in the storage device 7; Fig. 22(a) is an example of image pick up 

ranges on an inspection surface; Fig. 22(b) is complete image information for each object; 
and Fig. 22(c) illustrates divided images of the object. 

DESCRIPTION OF THE SPECIFIC EMBODIMENTS 
25 Hereinafter, an example of the present invention will be described with 

reference to the drawings. 

Figs. 1(a) and (b) show an example of a flaw to be inspected in accordance 
with the present invention. 

Fig. 1(a) shows an example of liquid penetrant inspection. An object 
30 under inspection 1 (hereinafter "object 1") is coated with white development liquid, and a 
flaw 2 (contrast: high) and a pseudo flaw 3 (contrast: low) are observed. In the liquid 
penetrant inspection, flaw 2 is enhanced and displayed as a red indication mark. The 
pseudo flaw appears as a pale red indication mark, such as when the penetrant has 
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remained in a surface grinding streak or the like and has not been completely polished 
out. 

Fig. 1(b) shows an example of magnetic particle testing. Fluorescent 
magnetic powder has been applied to object 1 having flaw 2, and the object has been 
5 magnetized. When the object is illuminated with ultraviolet light, the fluorescent 

magnetic powder collected at flaw 2 emits green light. However, if object 1 has, e.g., a 
welded portion, as the fluorescent magnetic powder collects along the bead, the pseudo 
flaw 3 may appear in green. 

Fig. 2 is a block diagram showing the construction of a flaw inspection 
10 apparatus according to the present invention. Object 1 has flaw 2 and pseudo flaw 3. 
Image pickup is performed on the object using a color video camera 21 . In liquid 
penetrant inspection, a white light 24a is used for illumination, while in magnetic particle 
testing, an ultraviolet light 24b is used for illumination. The white light 24a is connected 
to a white light connector 25a, and to an illumination power source 8 via an illumination 
1 5 cable 26. 

In magnetic particle testing, illumination cable 26 is connected to an 
ultraviolet illumination connector 25b. To avoid influence by extraneous light, a hood 27 
is attached to the light. In Fig. 2, the light is a ring-shaped light, however, one or more 
bar-shaped lights may be employed. 

20 A color video signal from the color video camera 21 includes independent 

R, G and B signals, or the signal may be a composite video signal. In any case, the video 
signal is stored as R, G and B image data in a color image memory 4. The color image 
data is analyzed by a computer 5, and the result of flaw detection is displayed on a color 
monitor 6. Computer 5 has a memory for storing a program and data, a processor for 

25 processing data detected by the color video camera 21, and an input/output interface 
connected to a keyboard (not shown) and to a color monitor 6. 

The flaw inspection result is stored in a data storage device 7. Further, the 
image displayed on color monitor 6 can be printed out, as needed. 

A polarizing filter 22a and an ultraviolet-light reduction filter 22b are 

30 attached to the front side of the lens of color video camera 21 . Further, a polarizing filter 
plate 23 is provided below white light 24a. Polarizing filter 22a and the polarizing filter 
plate 23 are used to prevent projection of white light and reflection from object 1 during 
liquid penetrant inspection. Polarizing filter 22a is rotated while an output video signal 
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from the color video camera 21 is monitored, and the polarizing filter is fixed at a 
position where the projection of illumination light and reflection are most effectively 
prevented. Polarizing filter 22a may be automatically adjusted based on the video output 
signal from the color video camera 21 . 
5 The ultraviolet light reduction filter is used for preventing unnecessary 

light emission from foreign materials and the like attached to object 1 when ultraviolet 
light 24b is used during magnetic particle testing. 

Fig. 3 illustrates the effects of the polarizing filter 22a and the polarizing 

filter plate 23. 

10 Fig. 3(a) shows a display on the monitor screen when the filters are not 

used; and Fig. 3(b), a display on the monitor screen when the filters are attached and the 
rotational angle of the filter is adjusted. In Fig. 3(a), projection of light 30 impedes flaw 
detection. The light is projected in a ring shape since white illumination light 24a is a 
ring type light. In Fig. 3(b), the projection does not occur. 

15 Figs. 4(a) and (b) illustrate the effect of the ultraviolet light reduction filter 

22b. Fig 4 (a) shows a display on the monitor screen when the filter is not used; and Fig. 
4(b), a display on the monitor screen when the filter is attached. In Fig. 4(a), light 
emission from a foreign material 40, such as lint, and positive reflection 41 from object 1, 
obtained by image pickup with color video camera 21, impede flaw inspection. In Fig. 

20 4(b), these noises are reduced, and the image shows light emission only from the 

fluorescent magnetic powder as in the case of visual observation of the product under 
inspection 1 by a human inspector. 

The method of crack flaw detection in a liquid penetrant inspection image 
will now be described with reference to Figs. 5 to 15. Fig. 5 shows an automatic 

25 detection method for flaw 2 in the liquid penetrant inspection. 

First, image pickup 50 is performed, using white light 24a, on object 1, 
which is coated with developer. Next, chromaticity conversion 51 is performed to obtain 
xy chromaticity values of the respective pixels from the R, G and B color image data. 

Next, determination of reference white 52 is performed to calculate the 

30 reference white chromaticity of the developer, and calculation of color hue and color 
difference 53 is performed in each position of the image with respect to the reference 
white. 
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Thereafter, to extract flaw candidate area 54, the area of color hue and 
color difference within a specific range is extracted by binarization. 

In most cases a true flaw 2 has a clear outline, and a pseudo flaw has an 
unclear outline. Accordingly, differentiation of color difference image 55 is performed to 
5 obtain the change rate for the color difference in the outline portion of the extracted flaw 
candidate area. Next, shape measurement 56 is performed to measure the area, the ratio 
of length to width, and the length of the flaw candidate area. Thereafter, flaw detection 
57 is performed to detect only an area having a high color difference change rate, and a 
length and area equal to or greater than the predetermined values for true flaw 2. Further, 

10 the inspection result is displayed on color monitor 6 for the inspector's flaw verification; 
then image data, shape data, positional information and the like are stored in storage 
device 7, or printed out and stored as a hard copy (58). 

Inspection by color requires quantitative color evaluation. Accordingly, at 
chromaticity conversion step 51, the RGB data of the obtained color image is converted 

15 to chromaticity x, y, and luminance Y defined by the CIE (Commission Internationale de 
TEclairage), and inspection is performed using the data. The two-dimensional orthogonal 
coordinate representation of the chromaticity x, y, as shown in Fig. 6 is called a 
chromaticity diagram. In the chromaticity diagram, colors are arranged around a white 
center, and the respective colors become more vivid as they get farther away from white. 

20 Hereinafter, a color tone will be called a color hue, and vividness of respective colors, 

saturation, and a distance between two chromaticity values on the chromaticity diagram, a 
color difference. Fig. 6 shows a color difference range 60 for a liquid penetrant 
inspection image. 

In a specific embodiment, to convert the RGB data to the chromaticity x, y 
25 and luminance Y with high precision, color calibration is performed in advance using a 
camera calibration color chart 71 as shown in Fig. 7. Fig. 8 shows the flow of the 
processing. Camera calibration color chart 71 is coated with three or more colors. It is 
subjected to image pickup by color video camera 21 (81), to calculate RGB values of the 
respective colors (82). Further, the chromaticity x, y and luminance Y of these colors are 
30 measured by a colorimeter 72 (83). The relation between RGB values and the xyY values 
is represented by (Expression 1) and (Expression 2). 
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(Expression 1) 



Chromaticity: 
Luminance: 



Note that X, Y and Z are called tristimulus values. 
X Y 



x =- 



X + Y + Z 



X + Y + Z 



(Expression 2) 



Accordingly, if the RGB values of the respective colors obtained from the 
camera are substituted into (Expression 1) and (Expression 2) to calculate the xyY values, 
and values an to a 33 corresponding to the xyY values measured by the colorimeter are 
obtained, conversion parameters unique to the camera can be obtained. Since the number 
of unknown parameters is 9, the parameters can be calculated by at least three color RGB 
values (R1G1B1) to (R3G3B3) and corresponding xyY values of the colorimeter (xiyiYi) to 
(x 3 y 3 Y 3 ). 

From (Expression 2), XYZ can be calculated from the xyY values by the 
following (Expression 3), 



X = Yxx/y, Y = Y, Z = Yx(l -x-y)/y 



(Expression 3) 



The xyY values of the three colors from the colorimeter are substituted 
into (Expression 3), to obtain XYZ, and the XYZ are substituted into (Expression 1). 
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(Expression 4) 



By this expression, conversion parameters an to a 33 unique to the camera 
are obtained 84, and the xyY values equal to those of the colorimeter can be obtained 
from the RGB values of the camera. 

For the camera calibration color chart 71 used in the liquid penetrant 
inspection, the arrangement of R, G, B and W reference colors and plural types of colors 
ranging from the white of the developer to the red of the penetrant is preferable for color 
calibration. The reproducibility of conversion parameters can be checked by selecting a 
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white which is close to that of the developer liquid, pink as the color of the flaw 
candidate, and red to correspond to a flaw, from a chromaticity range 60 of the liquid 
penetrant inspection image in Fig. 6, in a stepwise manner; and measuring the xyY values 
and comparing these values with respective xyY values calculated by using the previously 
5 generated conversion parameters for the xyY values unique to the camera. Reliable and 
high-precision chromaticity measurement can be easily realized by checking the color 
reproducibility periodically by using the color chips as shown in Fig. 6 in (or preferably 
prior to) liquid penetrant inspection. 

Further, the present inventors have observed that the reference colors of 
10 the above color chart differ depending on the color temperature of the illumination light 
source. That is, in liquid penetrant inspection with the white illumination light used in the 
embodiment of Fig. 2 as the light source, and, in liquid penetrant inspection with a 
halogen lamp as the light source. 

Since the chromaticity range in Fig. 6 may differ, the color calibration 
15 chromaticity must be selected in correspondence with the light source. 

Further, the chromaticity from white to red differs depending on the 
difference of the color tone of the surface of object 1 (a smooth bright surface of a metal 
such as stainless steel, a leather-like dark surface of iron or the like, a rusty brown surface 
etc.). Accordingly, the conversion to xyY can be performed with higher precision by 
20 performing color calibration with increased numbers of interpolation colors in white, pink 
and red. 

Further, it is preferable to select the chromaticity of the reference colors of 
red, green, blue, white, and colors changing from white to red, in correspondence with the 
chromaticity of the developer and the penetrant. 

25 When the RGB values obtained from the camera are chromaticity- 

converted to xyY values using the conversion parameters unique to the camera, as 
calculated by prior calibration, and the chromaticity distribution in the image is 
calculated, at step 52, calculation is performed using the chromaticity of the developer, 
i.e., the chromaticity of portions other than a flaw, as a reference value. First, the 

30 chromaticity values x, y of each pixel in the image are examined, and the number of 
pixels of each x, y value is counted; thus, a two-dimensional chromaticity histogram is 
generated as the graph of Fig. 9(a). 
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Then, the x chromaticity value of the largest number of pixels (Fig. 9(b)) 
and the y chromaticity value of the largest number of pixels (Fig. 9(c)) in the image are 
obtained. As most of the image portions include no flaw, the x, y chromaticity values of 
the peak value in the two-dimensional chromaticity distribution are the x, y chromaticity 
5 values of the reference white. 

At step 53 ? the color hue and color difference in each position on the 
image with respect to the reference white are calculated. Assuming that the chromaticity 
of the reference white is (x c , y c ), and the chromaticity in a position in the image is Xij, yij, 
the color hue in a position (i, j) is calculated in a direction toward the reference color on 
10 the chromaticity diagram as shown in Fig. 10. The calculation expression is as 
represented by (Expression 5). 



Color hue: 9y = tan" 1 



f _ \ 

y y y c 

JCij JC c 



(Expression 5) 



Further, as shown in Fig. 1 1, the color difference in the position (i, j) is 
calculated with a distance from the reference color on the chromaticity diagram. The 
15 calculation expression is as represented by (Expression 6). 
Color difference: 

dij = a/ (x& - xc) 2 + (yij - y c ) 2 (Expression 6) 

As shown in Fig. 12, from the color hue and color difference in each 
position of the image with respect to the reference white, calculated as above, the range to 

20 be detected as a flaw is limited in the color hue (in Fig. 12, the color hue stands in the 

range i < < 2 ), and the level of difference in vividness from the reference white is limited 
in the color difference (in Fig. 12, the color difference d stands in the range di < d < d 2 ). 
Then a portion within the range is extracted as a flaw candidate area. 

Among the flaw candidates obtained by limiting the range in the color hue 

25 and color difference in this manner, there is an area which is not to be detected as a flaw. 
For example, an area where the chromaticity gradually changes with respect to the 
reference white is not a flaw, but an area with a clear outline is a flaw. Accordingly, an 
area with smooth color change with respect to peripheral color is regarded as a normal 
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portion or the pseudo flaw 3, and only an area with radical change is regarded as the flaw 
2. At step 55, regarding a flaw candidate area, the change amount of color difference 
with respect to the reference white is obtained, and only an area where the change amount 
value is equal to or greater than a predetermined value is regarded as a flaw. 
5 Referring to Fig. 13(a), a flaw candidate area 131 extracted at step 54 is 

shown. Numeral 133 in Fig. 13(b) denotes a graph of color difference with respect to the 
reference white on 132 in Fig. 13(a). Further, the change amount of the color difference 
133 in each position on 132, i.e., differentiation of 133, is the color difference 
differentiation distribution 134 in Fig. 13(d). In this manner, in a position where the 
10 change amount of color difference with respect to the reference white is small, the 

differentiated value is small. As shown in Fig 13(d), only an area where the differentiated 
value is greater than a constant value 135 is regarded as a flaw area. As a result, only a 
flaw area 136 as in Fig. 13(c) where the color difference is large, i.e., the outline is clear, 
is detected. 

1 5 Next, the method for determination of the threshold value 135 will be 

described with reference to Fig. 14. In a graph of Fig. 14(a), the vertical axis is the 
maximum value of the color difference within each flaw candidate area extracted by the 
color hue and color difference, and the horizontal axis is the maximum value of the color 
difference differentiated value of the outline portion of each flaw area. The value of true 

20 flaw 2 is plotted by " X" and the value of the pseudo flaw 3, by "O" Further, 141a 
denotes a histogram of respective color difference differentiated values, and 142a, a 
histogram of color difference values. In a case where a flaw and a pseudo flaw are 
apparently distinguished from each other, a judgment line 144a is a straight line passing 
through the peak values of the histograms 141a and 142a and vertical to an inertial main 

25 axis 143a of the plotted points. Further, Fig. 14(b) illustrates a case where the flaw and 
the pseudo flaw are not separated, i.e., there is no peak or valley in the histogram. Here, 
the judgment line is 144b. That is, all the flaw candidate areas are detected as flaws, so as 
to prevent oversight. 

Next, the method for flaw detection by magnetic particle testing will be 

30 described with reference to Figs. 15 and 16. 

Fig. 15 shows an example of an image processing algorithm to analyze the 
content of image memory 7 in magnetic particle testing. An RGB image is read 151. 
Next, differentiation processing is performed on a G image with the greatest amount of 
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information on light emission from the fluorescent magnetic powder 152. By this 
processing, an area with a significant linear luminance change, such as a crack, is 
enhanced, while an area with high luminance but a small luminance change, such as 
accumulated magnetic powder, is not enhanced. 
5 Next, a binarization threshold value is determined from an average value 

of the G differentiated image, and binarization is performed 153. From the binarized 
image, image noise such as an isolated point is eliminated 154, so that flaw candidates are 
obtained. Then, the length, contrast and the like of these flaw candidates are calculated 
155. If these values are greater than predetermined values, the area is determined to be a 
10 flaw. 

Fig. 16 shows the method for discrimination between a flaw and a pseudo 
flaw. For example, as shown in Fig. 16(a), if the luminance distribution of flaw 2 and 
pseudo flaw 3 is plotted on a line 161, luminance distribution 162 as in Fig. 16(b) is 
obtained. The luminance values of flaw 2 and pseudo flaw 3 are approximately 

15 equivalent to each other. If luminance distribution 162 is differentiated, luminance 
differentiated distribution 163 as shown in Fig. 16(d) is obtained. In flaw 2, the 
luminance radically changes, while in pseudo flaw 3, it smoothly changes. By judging 
the result of differentiation processing by using a judgment threshold value 164 as shown 
in Fig. 16(d), only flaw 2 can be extracted as in Fig. 16(c). 

20 Referring to Fig. 17, flaw check and data storage will be described. The 

flaw and the pseudo flaw are separated by automatic inspection and only the flaw is 
extracted. However, to prevent oversight and misjudgment, in both liquid penetrant 
inspection and magnetic particle testing, a final flaw verification is performed by visual 
observation. 

25 Fig. 17 is a flowchart showing the flaw verification process. First, a 

portion judged by automatic judgment as a flaw is marker-displayed as a flaw candidate 

171 . Next, computer 5 requires the inspector to judge the flaw candidates one by one 

172. The inspector judges whether or not the candidate is a true flaw by viewing an 
original color image 173, and if the inspector judges that the candidate is a true flaw, the 

30 position, the length, the contrast and the like of the flaw are registered in data storage 
device 7 (174), and the marker is turned to red 175. 

In flaw candidate verification, if the inspector judges a candidate as a 
pseudo flaw the marker is deleted 176. If another flaw candidate exists, the marker is 
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displayed on the next flaw candidate. When all the flaw candidates have been checked 
177, the original color image is stored in data storage device 7 (178). 

Fig. 18 shows an example of a method for flaw candidate marker 
generation. In a flaw candidate 181, a central line 182 connecting a start point PI to an 
5 end point P2 is obtained, and longer sides AG and CD of a flaw candidate marker 1 83 are 
set in a position parallel to the central line and away from the line by a constant value m. 
Shorter sides AD and BC are similarly set. The length of flaw is the distance between the 
points PI and P2. In magnetic particle testing, contrast related to the depth of the flaw is 
obtained by scanning a contrast calculation line 184 from PI to P2, obtaining the 

10 difference between average luminance and maximum luminance on the line, obtaining the 
difference from PI to P2, and obtaining an average value of the difference values as the 
contrast of the flaw. Note that the flaw candidate marker is not limited to a rectangular 
shape. The shorter sides AD and BC may be semicircular lines. It is important that the 
flaw is not hidden by the marker. 

15 Fig. 19 shows an example of a flaw candidate display method using a color 

monitor 9. Verification is performed sequentially from a long candidate, using the 
original image. At first, all the markers are displayed in white; then a marker of a 
candidate judged as a true flaw is turned to another color, e.g., red, and a marker of a 
candidate judged as a pseudo flaw is deleted. 

20 Fig. 20 shows an example of a method for specifying an inspection 

position when the object under inspection 1 is a long object. A scale 201 with markings 
is fixed to the object under inspection 1, so that the scale 201 enters in a camera view 202. 
The markings of the scale are, e.g., numerals at intervals of one centimeter. Further, 
scale 201 for liquid penetrant inspection and the scale for magnetic particle testing may 

25 be in different colors. For example, in liquid penetrant inspection, scale 201 is a white- 
based scale with red markings and numerals, but in magnetic particle testing, scale 201 is 
a white-based scale with markings and numerals in green fluorescent color. 

Fig. 21 shows an example of an image obtained by image pickup. The 
image of scale 201 is simultaneously taken in a lower part of the image screen, and the 

30 camera position of object 1 is calculated from scale 201. That is, scale 201 has marking 
numerals 210 recognizable by the pattern matching method using the computer 5. 
Further, because scale 201 has tick lines 211 at intervals of, e.g., one centimeter, a more 
detailed camera position can be calculated. As an image signal of testing line 212 of CI to 
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C2 on the image, a cross section signal 213 is obtained. Then, positions of left end A and 
right end B, and El, E2, E3, E4 and E5 of tick lines 21 1 are obtained. The image pickup 
magnification can be calculated from the positions of El, E2, E3, E4 and E5, and with 
marking numerals 210, and the accurate position of flaw 2 on the object 1 can be 
5 obtained. 

The inspection result is stored in storage device 7. Figs. 22(a)-22(c) show 

an example storage format. 

When a surface to be inspected of the object 1 is large and the entire 

inspection surface is larger than one inspection image screen, the surface is divided into 
10 several images and image pickup and inspection are performed on the images. The image 

pickup ranges overlap with each other by a small amount on the inspection surface. 

Numerals 221a, 221b and 221c denote divided images of object 1. The inspection is 

performed on the respective divided images. As denoted by 222, the result is that 

complete information for the entire image of each object under inspection is stored, and 
15 further, information on the position, the length, the area, the chromaticity, the color hue, 

and the like of each flaw are stored with the image information. 

The inspector first displays the data 222 of each inspected object stored in 

storage device 7 on the display screen of monitor 6 and checks it. Further, if the inspector 

wants to see a portion of a flaw in detail, a corresponding divided image is read from the 
20 name of the object under inspection and its image number, and is displayed on the screen 

of monitor 6. At this time, the information on the position, the length, the area, the 

chromaticity and color hue, stored in correspondence with the displayed image data, can 

also be displayed on the screen of monitor 6. 

Further, as detected flaw candidates are displayed with enhancement using 
25 a marker or the like on the display screen, oversight of a flaw which is larger than a size 

of about 0.1 to 0.3 mm, about the same size as that in conventional visual observation 

inspection, can be prevented. 

Moreover, by increasing the image detection magnification, a flaw finer 

than a visually observable size can be detected. Further, by enlarging and displaying the 
30 flaw finer than a visually observable size on the display screen, the position, the length, 

the area, the chromaticity, the color hue and the like of the flaw can be checked on the 

display screen. 
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According to the present invention, image input is performed by using a 
color video camera, and in flaw inspection by magnetic particle testing, ultraviolet light 
reflected from an object under inspection can be reduced by the ultraviolet light reduction 
filter. Accordingly, the inspector can easily check the results of automatic flaw 
5 inspection. Moreover, because the flaw candidates are automatically indicated on the 
display screen, oversight in inspection can be almost fully prevented. Further, because 
the inspection image is stored, after the inspection, the stored image can be displayed on 
the display screen and the flaw can be checked again. Thus, the reliability of inspection is 
improved. 

1 0 Further, according to the present invention, because a the color video 

camera is used, the automatic flaw inspection by magnetic particle testing and liquid 
penetrant inspection can be performed by the same sensor probe. Thus, the convenience 
is greatly improved. 

The invention may be embodied in other specific forms without departing 

15 from the spirit or essential characteristics thereof. The present embodiment is, therefore, 
to be considered in all respects as illustrative and not restrictive, the scope of the 
invention being indicated by the appended claims rather than by the foregoing 
description, and all changes that come within the meaning and range of equivalency of 
the claims are therefore intended to be embraced therein. 

20 The preceding has been a description of the preferred embodiment of the 

invention. It will be appreciated that deviations and modifications can be made without 
departing from the scope of the invention, which is defined by the appended claims. 
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What is claimed is: 

1 LA flaw inspection method by magnetic particle testing, comprising: 

2 obtaining an image of a surface of a sample to be inspected by using a 

3 color video camera where magnetic particle is coated; 

4 extracting a flaw candidate of said inspected surface by processing the 

5 obtained image; 

6 displaying an image of the extracted flaw candidate on a display screen; 

7 and 

8 storing the displayed image to a memory. 

1 2. The flaw inspection method according to claim 1 , wherein the 

2 image stored in the memory is an image of a flaw extracted from the flaw candidate. 

1 3. The flaw inspection method according to claim 1 9 wherein the flaw 

2 candidate is detected by using information on luminance of a green (G) signal component 

3 among three RGB primary color signals from the color video camera. 

1 4. The flaw inspection method according to claim 1, wherein the 

2 image obtained includes a positional information and the positional information is 

3 displayed on the display screen with the image. 

1 5 . A flaw inspection method by liquid penetrant testing, comprising: 

2 illuminating a surface of a sample to be inspected with polarized light; 

3 obtaining an image of the surface illuminated with the polarized light; 

4 extracting a flaw candidate from the detected image of the surface by 

5 processing the obtained image; and 

6 displaying an image of the extracted flaw candidate. 
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1 6. The flaw inspection method according to claim 5, wherein a 

2 chromaticity of the obtained image is converted using a parameter. 

1 7. The flaw inspection method according to claim 6, wherein the 

2 image is detected by using a color video camera and the parameter for converting the 

3 chromaticity of the image is unique to the color video camera. 

1 8. The flaw inspection method according to claim 5, wherein the 

2 image displayed is associated with a positional information. 

1 9. The flaw inspection method according to claim 8, wherein the 

2 positional information is obtained with the image. 

1 10. The flaw inspection method according to claim 5, further 

2 comprising: 

3 detecting a flaw from the extracted flaw candidate; and 

4 storing an image of the detected flaw into memory. 

1 1 1 . A flaw inspection method, comprising: 

2 illuminating a surface of a sample to be inspected with light; 

3 obtaining an image of the surface; 

4 extracting a flaw candidate of the inspected surface by processing the 

5 obtained image; 

6 displaying an image of the extracted flaw candidate; 

7 detecting a flaw from the extracted flaw candidate; and 

8 storing an image of the detected flaw into memory. 

1 12. The flaw inspection method according to the claim 1 1, further 

2 comprising: 

3 re-displaying the stored flaw image. 

1 13. The flaw inspection method according to the claim 1 1 ? wherein the 

2 light illuminating the sample surface is polarized light. 

1 14. The flaw inspection method according to the claim 1 1 , wherein the 

2 light illuminating the sample surface is ultra violet light. 
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1 15. The flaw inspection method according to the claim 1 1, wherein the 

2 image is detected by using a color video camera. 

1 16. The flaw inspection method according to the claim 11, wherein the 

2 image is displayed on a screen together with a positional information, 

1 17, The flaw inspection method according to the claim 1 1, wherein a 

2 chromaticity of the image displayed is converted from the image using a parameter 

3 unique to the color video camera. 

1 18. The flaw inspection method according to the claim 1 1 , wherein the 

2 image of the flaw candidate is displayed on a screen distinguishable from others. 

1 19. A flaw inspection method, comprising: 

2 obtaining an image of a surface of an object to be inspected; 

3 detecting a flaw from the obtained image; 

4 displaying the detected flaw image on a screen; and 

5 storing the displayed flaw image in memory; wherein the flaw image is 

6 displayed on the screen distinguishable from other part of the object. 

1 20. The flaw inspection method according to the claim 1 9, further 

2 comprising: 

3 illuminating the object with a polarized light. 

1 21 . The flaw inspection method according to the claim 19, further 

2 comprising: 

3 illuminating the object with an ultra violet light. 

1 22. The flaw inspection method according to the claim 19, wherein the 

2 image is detected using a color video camera. 

1 23. The flaw inspection method according to the claim 1 9, wherein the 

2 detected flaw image is displayed on a screen together with a positional information. 

1 24. A flaw inspection apparatus based on flaw testing, comprising: 

2 illumination means for illuminating a surface of a sample to be inspected; 
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3 a color video camera which obtains an image of the surface; 

4 flaw candidate extraction means for extracting a flaw candidate of the 

5 surface from the image obtained by said color video camera; 

6 display means for displaying an image of the flaw candidate extracted by 

7 said flaw candidate extraction means; 

8 flaw detection means which detects a flaw from the displayed flaw 

9 candidate; and 

10 memory means for storing the image of flaw detected by the flaw detection 

1 1 means. 

1 25. The flaw inspection apparatus according to claim 24, wherein the 

2 display means displays the image of the flaw candidate accompanied with a positional 

3 information. 

1 26. A flaw inspection apparatus based on flaw testing, comprising: 

2 a light source which illuminates a surface of a sample to be inspected; 

3 a color video camera which obtains an image of the surface; 

4 a chromaticity converter which converts a chromaticity of the image 

5 obtained by the color video camera by using a conversion coefficient which is unique to 

6 the color video camera; 

7 a flaw candidate extractor which extracts a flaw candidate of the surface 

8 from the image obtained by said color video camera which chromaticity is converted by 

9 the chromaticity converter; 

10 a display which displays on a screen an image of the extracted flaw 

1 1 candidate which chromaticity is converted; 

12 a flaw detector which detects a flaw from the displayed flaw candidate; 

13 and 

14 a memory which stores the image of flaw detected by the flaw detector. 

1 27. The flaw inspection apparatus according to claim 26, wherein said 

2 chromaticity converter obtains conversion coefficients for converting RGB(Red, Green 

3 and Blue) chromaticity values unique to said color video camera into reference xy 

4 chromaticity values. 
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1 28. A computer memory storing code for a flaw inspection method 

2 using an object to be inspected, wherein said computer memory comprises: 

3 code for obtaining an image of a surface of the object; 

4 code for converting a chromaticity of the obtained image to extract a flaw 

5 candidate; 

6 code for detecting a flaw from the extracted flaw candidate; and 

7 code for displaying the detected flaw image on a screen. 

1 29. A computer memory storing code for a flaw inspection method 

2 using an object to be inspected, wherein said computer memory comprises: 

3 code for obtaining an image of a surface of the object; 

4 code for converting a chromaticity of the obtained image; 

5 code for displaying on a screen an image of the object which chromaticity 

6 is converted from the obtained image; 

7 code for indicating a flaw candidate on the screen; and 

8 code for displaying a flaw image on a screen detected from the candidate. 
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Nondestructive Inspection Method and an Apparatus Thereof 

ABSTRACT OF THE DISCLOSURE 
The present invention relates to a method for inspecting a crack in a metal 
surface or the like, and, particularly, to an inspection method and apparatus for 
nondestructive inspection such as liquid penetrant inspection and magnetic particle 
testing. The present invention provides a flaw inspection method that essentially 
comprises the steps of illuminating a surface of a sample to be inspected, obtaining an 
image of the surface, characterizing a potential flaw on the inspected surface by 
processing the obtained image, displaying an image of the potential flaw, verifying that 
the potential flaw is a true flaw, and storing an image of the verified flaw in memory. 
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